Data Quality and Medical Record Abstraction in the Veterans Health Administration's External Peer Review

or Outlier Report 2001 3rd Quarter QIC Outlier Report: Key Performance Indicators for QIC 199 variable level description Chg from history Chg from previous 2001 3Q 2001 2Q 2001 1Q 2001 0Q BL HXASCVD 1 Yes extreme increase significant increase 26% 17% 0.4% 4.3% 2 No extreme decrease significant decrease 74% 83% 100% 96% TOBSTATUS 1 Current user increased increased 19% 19% 18% 18% 2 Former user increased increased 22% 21% 17% 19% 3 Denies current use/ no further info decreased decreased 22% 22% 29% 22% 4 No use in past 7 years increased increased 35% 31% 28% 28% 5 No documentation extreme decrease significant decrease 2.3% 6.2% 8.8% 13% SPC Analysis of Increase in Depression Screening Techniques used to Assess Data Reliability Interrater Reliability Assessment Intrarater Reliability Assessment False Negative & False Positive Rates Service/Clinical Indicator Date Variance Item Reliability Assessment Interrater Reliability Assessments “IRRAs” occur between abstractors and either their field supervisor or an auditor Attempt to interrate between 20 and 25 records Calculation of agreement using weighted percent agreement and Kappa “beyond chance” agreement Abstractors (and items) yielding low Kappa agreement (< .85) are identified for QI training Problems with Kappa in Contexts of High Goal Attainment Prevalence of an observed trait: 100% Agreement that a service was provided = No Kappa Score Example: Yes No Yes 20 0 No 0 0 % agreement = 100 Kappa can not be calculated One disagreement can yield a Kappa Score of Zero 95%+ Agreement that a service was/was not provided can yield a zero or negative Kappa Score Example: Yes No Yes 19 1 No 0 0 % agreement = 95 Kappa = 0 Proceedings of the Sixth International Conference on Information Quality